cpi uiaii 



Making new connections towards cooperation in the prisoner's 
dilemma game 



oo 
o 
o 

(N 
> 

o 

6 
o 

O 

• i-H 

Oh 



> 



oo 
O 

> 

X 



Attila Szolnoki, 1 Matjaz Perc 2 and Zsuzsa Danku 3 



1 Research Institute for Technical Physics and Materials Science, P.O. Box 49, H-1525 Budapest, Hungary 

2 Faculty of Natural Sciences and Mathematics, University of Maribor, Koroska cesta 160, SI-2000 Maribor, Slovenia 



Faculty of Natural Sciences, Szeged University, H-6720 Szeged, Hungary 



PACS 02 . 50 . Le - Decision theory and game theory 
PACS 87 . 23 . Ge - Dynamics of social systems 

PACS 89 . 75 . Fb - Structures and organization in complex systems 

Abstract. - Evolution of cooperation in the prisoner's dilemma game is studied where initially 
all players are linked via a regular graph, having four neighbors each. Simultaneously with the 
strategy evolution, players are allowed to make new connections and thus permanently extend their 
neighborhoods, provided they have been successful in passing their strategy to the opponents. We 
show that this simple coevolutionary rule shifts the survival barrier of cooperators towards high 
temptations to defect and results in highly heterogeneous interaction networks with an exponential 
fit best characterizing their degree distributions. In particular, there exist an optimal maximal 
degree for the promotion of cooperation, warranting the best exchange of information between 
influential players. 



Introduction. — Complex networks, and scale-free 
networks in particular, have been identified as potent pro- 
moters of cooperation in all major types of social dilem- 
mas [1]. Especially the prisoner's dilemma, being one 
of the most widely applicable games [2], as well as the 
snowdrift and ultimatum games, have thus far been stud- 
ied on diluted [3,4] and hierarchical networks [5], ran- 
dom graphs [6], small- world [7-9] and real empirical net- 
works [10], as well as games on graphs in general [11]. Ar- 
guably the most important feature of complex networks 
responsible for the promotion of cooperation is the het- 
erogeneous linkage of participating players, which is due 
to large differences in the degrees of constitutive nodes. 
Indeed, studies have shown that masking the heterogene- 
ity via the introduction of participation costs or the usage 
of normalized or effective payoffs [12-15] eliminates exces- 
sive benefits for cooperators and yields results similar to 
those reported on regular square lattices [16-19]. Since 
heterogeneities due to complex interactions networks have 
proven wildly successful in promoting cooperation, similar 
characteristics have been introduced also via differences in 
the influence and teaching activities of players [20-22] and 
social diversity [23,24]. 

Besides introducing the relevant heterogeneities arti- 
ficially, recently it has been shown that the latter can 



emerge also spontaneously as a part of a coevolutionary 
process accompanying the evolution of strategies. Partic- 
ularly, in Ref. [25] the teaching activity was considered as 
an evolving property of players, and it has been shown that 
simple coevolutionary rules may lead to highly heteroge- 
neous distributions of teaching activity from an initially 
non-preferential setup, which in turn promotes coopera- 
tion in social dilemmas such as the prisoner's dilemma 
or the snowdrift game. Moreover, similar results were ob- 
tained by Poncela et al. [26] , who considered a coevolution- 
ary preferential attachment and growth scheme to gener- 
ate complex networks on which cooperation can thrive. 
An important precursor to these studies were works con- 
sidering the coevolution of strategy and structure [27] as 
well as random or intentional rewiring procedures [28-31], 
showing that they as well may help to maintain cooper- 
ative behavior. Interestingly, similar effects can also be 
observed if the players are allowed to move on the lattice 
during the strategy evolution [32]. 

In this letter, we propose a new model based on a sim- 
ple coevolutionary rule that, alongside the evolution of 
the cooperative and the defective strategy within the pris- 
oner's dilemma game, entails increasing the neighborhood 
of players by allowing them to make new permanent con- 
nections with the not yet linked neighbors. The only con- 
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dition necessary to exercise this is a successful pass of the 
player's strategy to one of its current opponents. Since 
each reproduction is considered as a statement of success 
of the donor player at that time, the latter is rewarded 
by the expansion of its neighborhood. Thus, the basic 
premise of the proposed coevolutionary rule is that in real 
social systems the more successful individuals will typi- 
cally have more associates than the less successful ones. 
Related to this, we study the impact of different limits 
with respect to the maximal degree an individual is al- 
lowed to obtain during the coevolutionary process, as well 
as the degree distribution of thereby resulting networks. 
Starting from a fully homogeneous and non-preferential 
setup where each player is linked only to its four near- 
est neighbors on a square lattice, we show that the model 
can sustain cooperation even at high temptations to de- 
fect, and moreover, that the resulting networks are highly 
heterogeneous with roughly an exponential degree distri- 
bution. In addition, we shed light on the observed phe- 
nomena by studying the interconnectedness of influential 
players and the information exchange between them. Our 
results suggest that 'making new friends' is an essential 
part of the evolutionary process, playing a crucial role by 
the sustainability of cooperation in environments prone to 
defection. 

The remainder of this letter is organized as follows. 
First, we describe the prisoner's dilemma game and the 
protocol for the coevolution of neighborhoods. Next we 
present the results, whereas lastly we summarize and dis- 
cuss their implications. 

Game definitions and setup. We consider an evo- 
lutionary prisoner's dilemma game with cooperation and 
defection as the two competing strategies. The game is 
characterized by the temptation to defect T = b, reward 
for mutual cooperation R = 1, and both the punishment 
for mutual defection P as well as the suckers payoff S 
equaling 0, whereby 1 < b < 2 ensures a proper pay- 
off ranking. Initially, each player x is designated either 
as a cooperator (C) or defector (D) with equal probabil- 
ity and linked to its four nearest neighbors on a regular 
L x L square lattice with periodic boundary conditions, 
thus having degree k = 4. This setup warrants that ini- 
tially all players have equal chances of success, which is 
crucial for evaluating the impact of the proposed coevolu- 
tionary rule. We note, however, that below results are ro- 
bust against variations in the initial conditions, as well as 
variations in the parametrization of the prisoner's dilemma 
game. The evolution of the two strategies is performed in 
accordance with the Monte Carlo simulation procedure 
comprising the following elementary steps. First, a ran- 
domly selected player x acquires its payoff p x by playing 
the game with all its k x neighbors. Next, one randomly 
chosen neighbor of x, denoted by y, also acquires its payoff 
p y by playing the game with all its k y neighbors. Last, if 
Px > Py player x tries to enforce its strategy s x on player 



y in accordance with the probability 

W(S X -> S y ) = (p x - Py)/bk q , (1) 

where k q is the largest of k x and k y . The introduction of k q 
is necessary since the degree k x is presently subject to evo- 
lution as well. In particular, each time player x succeeds in 
passing its strategy to player y the degree k x is increased 
by an integer AA; according to k x — > k x + Ak, whereby for 
simplicity we here use Afc = 1. Practically, the increase of 
degree k x is realized so that player x establishes a perma- 
nent new connection with a not yet connected player which 
is selected randomly amongst the direct neighbors of the 
current neighborhood of x. Thus, successful players are al- 
lowed to grow compact large neighborhoods that are cen- 
tered around their initial four nearest neighbors. Notably, 
similar results as will be reported below can be obtained 
if players extend their neighborhoods via long-range con- 
nections, but since we primarily wanted to eschew effects 
of resulting small-world topologies [7] and focus solely on 
the impact of coevolutionary extending neighborhoods, we 
here present the results obtained with the former model. 
The described coevolutionary rule would eventually result 
in a fully connected graph, which in turn would prevent 
the survival of cooperation due to the applicability of the 
well-mixed limit. Accordingly, to abridge the latter effect 
we introduce k max as the maximal degree a player is al- 
lowed to obtain. In fact, the coevolutionary process of 
making new connections is stopped as soon as the degree 
k of a single player within the whole population reaches 
k m ax, whereby this limit prevents the formation of a ho- 
mogeneous system and will be one of the main parameters 
to be varied below. Despite being strikingly simple, the 
proposed protocol for the coevolution of neighborhoods is 
remarkably robust, delivering conclusive results with re- 
spect to the final distribution of k as well as the two com- 
peting strategies. 

In accordance with the random sequential update, each 
individual is selected once on average during a full Monte 
Carlo step (MCS), which consists of repeating the above 
elementary steps L 2 times corresponding to all partici- 
pating players. Monte Carlo results were obtained on 
populations comprising 100 x 100 to 400 x 400 individ- 
uals, whereby the stationary fraction of cooperators pc 
was determined within 10 5 to 10 6 MCS after sufficiently 
long transients were discarded. Moreover, as the coevo- 
lutionary process yields highly heterogeneous interaction 
networks, thus yielding heavily fluctuating outputs, final 
results were typically averaged over 200 independent runs 
for each set of parameter values in order to take into ac- 
count the stochastic feature of host graph topology result- 
ing from the coevolutionary process. 

Results. — The remarkable impact of the above de- 
fined coevolutionary process is demonstrated in Fig. 1, 
where the time evolution of pc obtained with and without 
the inclusion of coevolution is presented. The difference 
between the two outcomes is evident as the basic version 
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Fig. 1: Time evolutions of pc obtained for b — 1.24. Dashed 
line shows results obtained in the absence of the coevolutionary 
neighborhood growth, while the solid line depicts the outcome 
of the prisoner's dilemma game if k ma x = 50. Note that the 
x-axis has a logarithmic scale on which fractions of the first 
full MCS are shown as well for clarity. 



of the game fails to sustain cooperative behavior (dashed 
line) while the inclusion of the coevolutionary process is 
able to recover it and maintain respectable pc = 0.66 
(solid line). Clearly thus, the newly proposed model is 
able to sustain cooperation in regions of b where the regu- 
lar square lattice interaction topology fails. However, it is 
interesting to note that this is true for the final outcome of 
the game, whereas during the first 100 MCS it seems that 
the cooperative behavior will actually fare better without 
the inclusion of the coevolutionary process. Note that the 
solid line drops slightly faster than the dashed line dur- 
ing the initial phase of the game. Yet rather surprisingly, 
the tide then shifts in favor of the cooperative strategy as 
depicted by the solid line in Fig. 1; a feature that cannot 
be observed in case the coevolution of neighborhoods is 
absent. Figure 1 also suggests that, while the results are 
essentially robust against variations of initial conditions, 
the initial density of cooperators shouldn't be too low, as 
otherwise the promotive impact of coevolution could be 
missed. 

To sharpen the facilitative effect of the coevolutionary 
rule on the cooperative behavior, we present stationary 
values of pc in dependence on k max for three different 
values of b in Fig. 2. It can be inferred that there exist 
an optimal maximal degree k max a player is allowed to 
obtain during the coevolutionary process at which coop- 
eration thrives best. This holds irrespective of 6, although 
the optimal values of k max fluctuate between 50 and 70. 
The non-monotonous dependence on k maxi illustrated in 
Fig. 2, is a consequence of the limited support for cooper- 
ation offered by the square lattice (results obtained at val- 
ues of kmax equal or close to 4), and the well- mixed limit 
that is reached for high k max . Note that k max comparable 
with the system size inevitably lead to a high degree of 
intcrconncctcdncss amongst the players, which is charac- 
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Fig. 2: Promotion of cooperation in dependence on k max for 
b = 1.15 (open squares), b — 1.2 (closed squares), and b = 1.28 
(open circles). There exists an optimal value of kmax at which 
pc is maximal. Lines are solely guides to the eye. 

teristic for a well-mixed population. In between the two 
extremes, the coevolutionary rule obviously yields a favor- 
able host graph topology for the evolution of cooperation. 

Figure 3 shows pc in dependence on b for three dif- 
ferent values of k max , whereby it can be observed that 
the optimal value of k max — 50 is able to sustain some 
fraction of cooperators almost halfway through the whole 
span of b. By comparison, in the absence of the coevo- 
lutionary process (note that k max = 4 leaves the initial 
topology unaltered) the cooperative trait goes extinct at 
b = 1.115. Moreover, large values of k max still yield some 
advantages for the cooperators, as can be inferred from the 
kmax = 200 curve depicted in Fig. 3, yet increasing k m ax 
even further introduces wcll-mixcd-likc conditions where 
the sustainability of cooperation is practically absent even 
for low values of b. 

Before turning our attention to networks emerging due 
to the proposed coevolutionary rule, we test above results 
against the separation of time scales [33], presently char- 
acterizing the evolution of strategies and structure. Thus 
far, the two time scales were treated as identical since ev- 
ery successful reproduction was followed by an increase 
in the player's degree. The model can be generalized 
via a parameter q that determines the probability of de- 
gree extension after a successful strategy pass. Evidently, 
q = 1 recovers the originally proposed model while de- 
creasing q result in increasingly separated time scales. At 
q = the model becomes equivalent to the spatial model 
without coevolution, hence yielding pc — by high b, as 
demonstrated in Fig. 4. An increase in q, resulting in a 
moderately fast yet effective coevolution, is beneficial for 
cooperation since influential cooperators can then extend 
their neighborhoods and thus become stronger by collect- 
ing higher payoffs already during the coevolutionary pro- 
cess. Oppositely, influential defectors become weaker as 
their defecting neighborhoods grow, which ultimately re- 
sults in the highest cooperation levels at intermediate q. 
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Fig. 3: Promotion of cooperation in dependence on b for 
kmax = 4 (open squares), k max = 50 (closed squares), and 
kmax = 200 (open circles). Cooperators are most successful if 
kmax = 50, which roughly corresponds to the peak values of 
pc depicted in Fig. 2. Lines are solely guides to the eye. 



However, further increasing q can generate a slight down- 
ward trend of pc because the influential cooperators can- 
not take full advantage of their newly acquired neighbors 
within the short time between consecutive building steps, 
and thus defectors can gain a slight yet permanent advan- 
tage. The moderate decrease in pc due to the too fast 
network evolution if compared to the strategy evolution 
is, however, virtually absent by very high b, since then 
the dominating feature is the final heterogeneous network 
topology rather than initial fights of dominance. For sim- 
plicity, and to preserve comparability with the results in 
the first three figures, we will continue to use q = 1 in 
what follows. 

Next, we examine properties of networks resulting from 
the coevolutionary process. As heterogeneity is the most 
important property favoring cooperation, we first focus 
on the degree distribution P(k). Given the fact that sub- 
stantial promotion of cooperation was in the past often 
associated with strongly heterogeneous states, either in 
form of the host network [1] or social diversity [23,24], it 
is reasonable to expect that P(k) will exhibit similar fea- 
tures as well. Results presented in Fig. 5 clearly attest 
to this expectation as the semi-log plot of the distribu- 
tion reveals a highly heterogeneous outlay of P(k) that 
can be most accurately described by an exponential fit. 
The latter feature is crucial for the fortified facilitative 
effect on cooperation outlined in Figs. 2 and 3, in par- 
ticular since it incubates cooperative clusters around in- 
dividuals with high k, as described previously in [1] and 
reviewed in [11]. Contrary, since the positive feedback of 
the imitating environment is not associated with influen- 
tial defectors they therefore fail to survive even if temp- 
tations to defect are large. As already noted, a similar 
behavior underlies the cooperation-facilitating mechanism 
reported for the scale-free network where players with the 
largest connectivity (presently equivalent to those having 
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Fig. 4: Cooperation level pc in dependence on the time sepa- 
ration between strategy and structure updating q for b = 1.2 
(open squares), b = 1.25 (closed squares), and 6 = 1.3 (open 
circles). The maximal degree was limited to k max = 50 for all 
three values of b. Lines are solely guides to the eye. 



k close to k max ) also act as robust sources of coopera- 
tion in the prisoner's dilemma game. Noteworthy since 
it is related to time courses presented in Fig. 1, before 
the heterogeneous network topology fully evolves, defec- 
tors temporarily thrive since they gain a larger base of 
neighbors to exploit. Once, however, the prime spots of 
the evolved network are overtaken by cooperators the de- 
fectors start loosing ground fast, which explains the initial 
drop and the subsequent recovery of cooperative behavior 
depicted by the solid line in Fig. 1. The presently re- 
ported spontaneous emergence of the heterogeneous dis- 
tribution of degree from an initially non-preferential state 
within the framework of evolutionary game theory sug- 
gests that even very simple coevolutionary rules might 
lead to a strong segregation amongst participating players, 
which is arguably advantageous for flourishing cooperative 
states. We argue that the core mechanism responsible for 
the emergence of heterogeneity in the degree distribution 
presented in Fig. 5 can be related to the growth and pref- 
erential attachment mechanism proposed by Barabasi and 
Albert [34]. In particular, our model incorporates pref- 
erential attachment in that the probability of increasing 
the degree is larger by players that have had successful 
reproductions in the past since they are more likely to 
reproduce in the future. Obviously, however, our model 
does not incorporate growth since players arc not added 
in time. Nevertheless, since the evolution is halted by a 
given k max , preferential attachment alone can still lead to 
highly heterogeneous but not scale- free distributions [34]. 

One may argue, however, that similar highly heteroge- 
neous degree distributions can be obtained at higher k max 
as well, yet the promotion of cooperation is then still mod- 
erate, as demonstrated in Fig. 2. This observation high- 
lights that the heterogeneous distribution itself is not a 
sufficient condition for ample levels of cooperation at high 
temptations to defect. To uncover the additional decisive 
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Fig. 5: Final distribution of degree P(k) in the studied pris- 
oner's dilemma game obtained for b = 1.26 via k max = 50. 
Note that the y axis has a logarithmic scale to clearly reveal 
the heterogeneous outlay of P(k). 



feature of resulting networks by different k max , we study 
the overlap of neighborhoods of the so-called influential 
players, whereby a player is designated as influential if it 
has the highest degree among any other players that can 
adopt the strategy from the influential player via an ele- 
mentary process. Figure 6 shows typical distributions of 
influential players, which are denoted either green (coop- 
erators) or red (defectors) depending on their strategy. In 
addition, their neighborhoods, formed by those directly 
linked to the influential players, are depicted yellow. The 
distributions are plotted for different values of k max but 
for an identical temptation to defect equalling b — 1.2. 
At small k max there exist many influential players with 
small neighborhoods surrounding them, yet the outlay is 
virtually homogenous, and thus the promotion of cooper- 
ation is not notably enhanced if compared to the square 
lattice alone. Around the optimal k max , however, influen- 
tial players become fewer, and their neighborhoods larger. 
Importantly though, the overlap between their neighbor- 
hoods is still remarkable, which is crucial as it enables 
influential cooperators to overtake influential defectors as 
soon as the latter weaken their neighborhoods. As we will 
show next, this effective information transfer between the 
influential players is crucial for the feedback mechanism 
to work. At higher k max influential players become rarer 
still, and their neighborhoods grow further, yet crucially, 
the overlap between them vanishes, thus hindering influen- 
tial cooperators to overtake influential defectors. In sum, 
defectors are virtually undisturbed in exploiting their large 
neighborhoods, ergo leading to a population in which de- 
fection is widespread. 

The impact of the above discussed topological feature 
can be studied directly by measuring the information 
transfer between influential players, which we realize via 
as quantifying the frequency of strategy adoptions be- 
tween influential players in the stationary state of the pris- 
oner's dilemma game for each value of k max . Figure 7 
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Fig. 6: Snapshots of typical distributions of players on a 
100 x 100 grid, obtained for k ma x = 14 (left panel), k max = 50 
(middle panel) and k max = 200 (right panel) at 6 = 1.2. Red 
and green are influential players (see text for details) in de- 
fector and cooperator states, respectively, while yellow are all 
the direct neighbors of the depicted influential players. If a 
player is neither influential nor belonging to a neighborhood of 
an influential player it is marked white. 

features the results, and in addition, shows the coopera- 
tion level for the sake of comparison. Note that both as 
and pc are depicted normalized with their maximal val- 
ues (denoted as [as] and [pc] on the vertical axis), yet the 
outlay of the curves thereby remains unaltered. At values 
of k max that are comparable to the initial degree of all 
participating players the neighborhood size of influential 
individuals is small. Thus, they cannot communicate effi- 
ciently with one another, which ultimately results in low 
values of as- As the maximally attainable degree limit in- 
creases the average neighborhood size of influential play- 
ers grows as well. Consequently, direct strategy adoptions 
between them become more frequent, and most crucially, 
strong influential cooperative players can overtake weak- 
ened influential defectors, in turn allowing the feedback 
mechanism to blossom in the intermediate region of k max . 
Contrary, when k max exceeds the optimal value some play- 
ers become too influential and grow well separated neigh- 
borhoods (see the right panel of Fig. 6), which hampers 
the information exchange between them so that influential 
defectors can prevail for prolonged periods of time despite 
of their weak posture inflicted by the defecting neighbor- 
hoods. Indeed, results presented in Fig. 7 show that the 
level of information exchange, quantified via as, and the 
global cooperation level in a heterogeneous environment 
are strongly bound to one another, following very similar 
patterns depending on k max , thus validating our reason- 
ing. Moreover, it is worth noting that our explanation is in 
agreement with a previous observation of Rong et al. [35] , 
who detected a fall of overall cooperation levels in the 
prisoner's dilemma game on scale-free networks following 
disassortative mixing, which also results in an enhanced 
isolation of hubs. 

Summary. — We have demonstrated that the intro- 
duction of a very simple coevolutionary process to the spa- 
tial prisoner's dilemma game markedly improves survival 
chances of cooperators in highly defection-prone environ- 
ments, and moreover, enhances their overall dominance at 
moderate temptations to defect. Most notably, the under- 
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Fig. 7: Frequency of strategy adoptions between influential 
players as (closed circles) and the cooperation level pc (open 
circles) in dependence on k max for b — 1.28. Both quantities 
are normalized by their maximal values for better comparisons, 
and are therefore decorated by square brackets on the corre- 
sponding axis label. Lines are solely guides to the eye. 

lying mechanism behind the reported promotion of coop- 
eration is routed in the resulting highly heterogeneous net- 
work structure which emerges spontaneously from a non- 
preferential setup following a simple coevolutionary rule 
that indirectly promotes players that are able to pass their 
strategy by allowing them to extend their neighborhoods 
via new connections to not yet linked players. Moreover, 
we have shown that the newly introduced coevolutionary 
rule yields optimal results if limited by a maximal degree 
the most influential player is allowed to obtain. If this 
limit is surpassed, the detrimental impact on the evolu- 
tion of cooperation sets in due to the decreasing overlap 
of neighborhoods of the influential players, which enables 
defectors to reign in isolation from potentially stronger 
influential cooperators. The success driven increase of de- 
gree indirectly introduces a preferential attachment mech- 
anism into the model, which combined with the limiting 
kmax, results in heterogeneous degree distributions. 

In sum, presented results confirm that the presence of 
influential leaders is advantageous for cooperation, and 
more importantly, that a simple 'making new friends' co- 
evolutionary rule may bring about just the appropriate 
diversity between participating players if appropriately 
timed. The coevolutionary model demonstrates how influ- 
ential leaders can evolve from an initially non-preferential 
state, and that it is optimal for cooperation if their overall 
density remains bounded to intermediate levels. 

* * * 

Discussions with Gyorgy Szabo are gratefully acknowl- 
edged. 
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